Imaging apparatus for use in a robotic surgery system

ABSTRACT

A stereoscopic imaging apparatus for use in a robotic surgery system is disclosed and includes an elongate sheath having a bore. First and second image sensors are adjacently mounted at the distal end to capture high definition images from different perspective viewpoints for generating three-dimensional image information. The image sensors produce an unprocessed digital data signal representing the captured images. A wired signal line transmits the unprocessed digital data signals along the sheath to a proximal end to processing circuitry. The processing circuitry is configured to perform processing operations on the unprocessed digital data signals to produce respective video signals suitable for transmission to a host system or for driving a 3D display. A secondary camera is also disclosed and includes an elongate strip of circuit substrate sized for insertion through a narrow conduit, the strip of circuit substrate connecting between an image sensor and a processing circuit substrate.

BACKGROUND 1. Field

This disclosure relates generally to imaging and more particularly to astereoscopic imaging apparatus for use in generating images within abody cavity of a patient.

2. Description of Related Art

Stereoscopic imaging generally involves capturing a pair of images fromspaced apart perspective viewpoints and processing the images togenerate a three-dimensional (3D) view or 3D information based on adisparity between the pair of images. Conventional endoscopes used inmedical and surgical procedures use relay lenses to convey capturedlight from a distal end of a narrow elongate tube inserted into apatient's body cavity to form an image at a proximal end of the tube.Alternatively, a small format image sensor capable of generating highresolution video signals may be used to capture an image at the distalend of the tube and to relay an image signal back to a host system fordisplay. When implementing high definition imaging at high frame rates,image signals that are transmitted back to the host system by the imagesensor have a relatively high data rate and there is consequentlysignificant heat generation at the image sensor. The heat generation atthe end of the tube may cause an unacceptable and/or unpredictabletemperature increase of the distal end of the endoscope and within thebody cavity of a patient.

SUMMARY

In accordance with one disclosed aspect there is provided a stereoscopicimaging apparatus for use in a robotic surgery system. The apparatusincludes an elongate sheath having a bore extending therethrough. Thesheath terminates in a distal end sized for insertion into a body cavityof a patient. First and second image sensors are adjacently mounted atthe distal end of the sheath and oriented to capture high definitionimages of an object field from different perspective viewpoints forgenerating three-dimensional image information. Each of the first andsecond image sensors is configured to produce an unprocessed digitaldata signal representing the captured images. The apparatus alsoincludes a wired signal line connected to transmit each of theunprocessed digital data signals from the first and second image sensorsalong the sheath to a proximal end thereof. The apparatus furtherincludes processing circuitry disposed at the proximal end of the sheathand connected to the wired signal line to receive the unprocesseddigital data signals from each of the first and second image sensors.The processing circuitry is configured to perform processing operationson each of the unprocessed digital data signals to produce respectivevideo signals suitable for transmission to a host system for driving adisplay capable of three-dimensional information.

Each of the unprocessed digital data signals may have a bit rate higherthan about 1 gigabit per second.

Each of the first and second image sensors have at about least 2,000,000pixels.

The unprocessed digital data signal may include 10 bit pixel intensityvalues read out from the pixels of the respective first and second imagesensors.

The unprocessed digital data signal may include a signal in accordancewith a MIPI Camera Serial Interface protocol and the length of thesheath may be greater than 30 millimeters.

The apparatus of the length of the sheath may be at least about 800millimeters.

The wired signal line may include a plurality of individual conductorsincluding conductors for implementing at least one MIPI data lane foreach image sensor, conductors for transmitting a synchronization clocksignal between the processing circuitry and the first and second imagesensors, and at least two conductors for carrying image sensor controlsignals.

The first and second image sensors may be mounted on a sensor circuitsubstrate disposed within the bore of the sheath and the wired signalline may include a plurality of individual conductors connected via thea sensor circuit substrate to unprocessed digital data outputs of therespective first and second image sensors.

The plurality of individual conductors of the wired signal line may beconnected at the proximal end to a strip of circuit substrate sized topass through the bore of the sheath, the strip of circuit substrateincluding a multiple pin connector for connecting to a correspondingmultiple pin connector on a circuit substrate associated with theprocessing circuitry.

The apparatus may include a graphene sheet within the bore of thesheath, the graphene sheet being in thermal communication with thesensor circuit substrate and wrapped around at least a portion of alength of the wired signal line for channeling heat away from the distalend of the sheath.

The apparatus may include a heating element disposed at the distal endof the sheath and operably configured to selectively heat the distal endof the sheath to maintain the distal end of the sheath at a temperaturethat prevents formation of condensation.

The apparatus may include signal conditioning circuitry for conditioningthe unprocessed digital data signals for transmission, the signalconditioning circuitry including at least one of conditioning circuitryat the distal end of the sheath between each of the first and secondimages sensors and the wired signal line, conditioning circuitry locatedpartway along the sheath in-line with the wired signal line, orconditioning circuitry configured to re-condition the receivedunprocessed digital data signals prior to performing processingoperations on the signals.

The processing circuitry may include circuitry that converts each of theunprocessed digital data signals into a serial digital interface (SDI)video signal for transmission to a host system.

The processing circuitry may include circuitry that converts each of theunprocessed digital data signals into a FPD link video signal fortransmission to a host system.

The sheath may include one of a rigid sheath or a flexible sheath.

The sheath may include a flexible articulating portion which whenactuated by the host system facilitates movement of the distal end ofthe sheath within the body cavity of a patient to orient the imagesensors for image capture.

The apparatus may include a plurality of optical fibers extendingthrough the sheath and terminating at the distal end, the plurality ofoptical fibers being operable to channel light from a distally locatedlight source for illuminating the object field.

The first and second image sensors may be mounted on a sensor circuitsubstrate sized to occupy a central portion of the bore of the sheathand the plurality of optical fibers may terminate at regions between thesensor substrate and the sheath at the distal end of the sheath.

The sheath may have a generally circular cross section.

The sheath may have an outside diameter of less than about 10millimeters.

Each of the image sensors may include imaging optics disposed in frontof the respective faces of each of the image sensors and configured tocapture light from the object field to form an image on the respectiveimage sensors.

In accordance with another disclosed aspect there is provided an imagingapparatus. The apparatus includes an image sensor oriented to capturehigh definition images of an object field and configured to produce anunprocessed digital data signal representing the captured images. Theapparatus also includes an elongate strip of circuit substrate sized forinsertion through a narrow conduit. The image sensor is mounted at adistal end of the circuit substrate and connected to a plurality ofconductors extending along the elongate circuit substrate to a proximalend thereof. The proximal end has a multiple pin connector forconnecting to a corresponding multiple pin connector on a processingcircuit substrate. The processing circuit substrate includes processingcircuitry configured to receive and process the unprocessed digital datasignal from the image sensor to produce a video signal suitable fortransmission to a host system for driving a display.

The elongate strip of circuit substrate may have a length of at leastabout 20 centimeters and a width of less than about 4 millimeters.

In accordance with another disclosed aspect an insertion device for arobotic surgery apparatus includes an insertion section including firstand second camera channels and at least one instrument channel extendingalong at least a portion of the insertion section. The first camerachannel is configured to facilitate insertion and removal from theinsertion section of the sheath and the first and second image sensorsof as disclosed above for use as a primary camera. The second camerachannel is configured to enclose the image sensor and elongate strip ofcircuit substrate disclosed above for use as a secondary camera. The atleast one instrument channel is configured to permit insertion andremoval of at least one surgical instrument from the insertion section.The apparatus also includes a housing attached to the insertion section.The housing includes a passage configured to permit at least a portionof the primary camera to pass through the housing into the first camerachannel and exit the first camera channel, the housing configured to beremovably attached to the robotic surgery apparatus. The secondarycamera is configured to provide image data of a surgical site tofacilitate insertion into the surgical site of at least one of the atleast one surgical instrument or the primary camera.

Other aspects and features will become apparent to those ordinarilyskilled in the art upon review of the following description of specificdisclosed embodiments in conjunction with the accompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

In drawings which illustrate disclosed embodiments,

FIG. 1 is a perspective view of a stereoscopic imaging apparatusaccording to a first disclosed embodiment;

FIG. 2A is a perspective view of first and second image sensors of thestereoscopic imaging apparatus shown in FIG. 1;

FIG. 2B is an exploded view of a sensor circuit substrate shownseparated from a connector circuit substrate of the stereoscopic imagingapparatus shown in FIG. 1;

FIG. 2C is a rear view of the connector circuit substrate shown in FIG.2B;

FIG. 2D is a further perspective view of first and second image sensorsof the stereoscopic imaging apparatus shown in FIG. 1;

FIG. 3 is a schematic diagram of processing circuitry, the connectorcircuit substrate, and the sensor circuit substrate of the stereoscopicimaging apparatus shown in FIGS. 1, 2A and 2B;

FIG. 4 is a perspective view of an insertion device through which thestereoscopic imaging apparatus shown in FIG. 1 is inserted; and

FIG. 5 is a perspective view of a secondary camera used in the insertiondevice shown in FIG. 4.

DETAILED DESCRIPTION

Referring to FIG. 1, a stereoscopic imaging apparatus according to afirst disclosed embodiment is shown generally at 100. The apparatus 100includes an elongate sheath 102. The sheath 102 terminates in a distalend 104 sized for insertion through an opening into a body cavity of apatient. The opening may be an incision made by a surgeon to permitaccess to the body cavity. In other embodiments the opening may be anatural orifice that permits access to the body cavity of the patient.The apparatus 100 may be part of a robotic surgery system for performingrobotic surgery. In one embodiment the sheath 102 may be about 800millimeters in length but in other embodiments may be longer or shorter.

The apparatus 100 further includes first and second image sensors at thedistal end 104 of the elongate sheath 102. The distal end 104 is shownin greater detail in the insert in FIG. 1 with a portion of the sheath102 removed. The first and second image sensors 106 and 108 areadjacently mounted at the distal end 104 of the sheath 102 within asensor housing portion 140 at the end of the sheath. The first andsecond image sensors 106 and 108 are oriented to capture high definitionimages of an object field 110 from different perspective viewpoints 112and 114 for generating three-dimensional (3D) image information.

The first and second image sensors 106 and 108 are each configured toproduce an unprocessed digital data signal representing the imagescaptured from the perspective viewpoints 112 and 114. Unprocesseddigital video signals generally represent the actual intensity read outfrom each pixel on the image sensor. Some image sensors are configuredto compress the video signal using a lossy compression method in whichsome of the video information may be removed. Otherwise known as araw-video signal, an unprocessed digital data signal maintains the fullintegrity of the actual image information and preserves options forsubsequent processing by the host system.

The apparatus 100 also includes a bore 116 extending through theelongate sheath 102 that accommodates a wired signal line 118 (shown inpart in the insert of the distal end 104). The wired signal line 118 isconnected to transmit each of the unprocessed digital data signals fromthe first and second image sensors 106 and 108 along the sheath 102 to aproximal end 120 of the sheath. The apparatus 100 also includesprocessing circuitry 122 disposed at the proximal end 120 of the sheath102. The processing circuitry 122 is shown in greater detail in theinsert in FIG. 1 and is connected to the wired signal line 118 via astrip of circuit substrate 124. Individual conductors of the wiredsignal line 118 are soldered to the strip of circuit substrate 124. Theprocessing circuitry 122 includes a multiple pin connector 126 thatconnects to a corresponding multiple pin connector on the strip ofcircuit substrate 124. The strip of circuit substrate 124 is sized topass through the bore 116 of the sheath 102 to facilitate threading ofthe wired signal line 118 through the sheath 102 from the distal end104. The processing circuitry 122 thus receives the unprocessed digitaldata signals from each of the first and second image sensors 106 and 108via the wired signal line 118 and is configured to perform processingoperations on each of the unprocessed digital data signals to producerespective video signals suitable for transmission to a host system fordriving a display (not shown) capable of displaying 3D information.

An advantage provided by the apparatus 100 is that the processingcircuitry 122 is separated from the distal end 104 of the sheath 102 andthe image sensors 106 and 108. The distal portion of the sheath 102 willgenerally be inserted into the patient's body cavity while theprocessing circuitry 122 remains outside the body cavity or otherwiseaway from the surgical site. Heat generated by the processing circuitry122 while processing and transmitting the image signals is thus able todissipate outside the body cavity of the patient (or otherwise away fromthe surgical site). Some heat is also generated by the first and secondimage sensors 106 and 108 but causes a lesser temperature increase thanwould be if the heat generated by the processing circuitry 122 were alsoto be dissipated proximate the distal end 104 of the sheath 102.

In the embodiment shown, the distal end 104 of the sheath 102 includes aflexible articulating portion 128, which includes a plurality ofvertebra 130 that are moveable when actuated by a host system (notshown) by pushing and/or pulling on a plurality of control links 132.The flexible articulating portion 128 is shown in the distal end insertwith one of the vertebra 130 omitted to reveal the underlying structure.In the embodiment shown, each vertebra 130 has a central opening forreceiving a tube 134 that defines the bore 116 within the sheath 102.The plurality of control links 132 are routed through respectivechannels extending through the vertebrae and the distal ends of thecontrol links are fixed at a last vertebra 136 in the flexiblearticulating portion 128. A face 138 of each vertebra includes a curvedportion to accommodate movement with respect to adjacent vertebra sothat the flexible articulating portion 128 is able to flex in alldirections. The plurality of control links 132 are shown truncated inthe insert of the distal end 104 but in practice extend through thelength of the sheath 102 and are connected to an actuator drive of thehost system at the proximal end 120. The vertebrae 130 move with respectto each other when actuated by the plurality of control links 132 causemovement of the distal end 104 of the sheath 102 such that the first andsecond image sensors 106 and 108 are oriented within the body cavity ofthe patient for image capture within the object field 110. In theembodiment shown, at least a portion of the tube 134 that passes throughthe flexible articulating portion 128 would be fabricated from aflexible material. However in some embodiments the entire tube 134 andthe sheath 102 may be fabricated from flexible materials that allow theapparatus 100 to be flexed along its length.

In the embodiment shown, the apparatus 100 and sheath 102 have agenerally circular cross section, and in one embodiment may have anoutside diameter of less than about 10 millimeters. In the embodimentshown, the apparatus 100 also includes a fiber bundle 142 including aplurality of optical fibers. The fibers insert in an outer perimeterspace 144 between the bore 116 and the sheath 102 at the proximal end120 and are routed through the sheath to the distal end 104 where thefibers terminate in regions 146 and 148 above and below the imagesensors 106 and 108. The fiber bundle 142 has an end 150 that couples toa distally located light source (not shown) that generates and coupleslight into the fiber bundle. The fiber bundle 142 guides the light alongthe sheath 102 to the regions 146 and 148 where the light is directed toilluminate the object field 110 for capturing images at the imagesensors 106 and 108. In other embodiments, the fibers may terminate inother regions at the distal end 104, including, for example, at aplurality of regions.

Referring to FIG. 2A, the first and second image sensors 106 and 108 areshown with the sensor housing portion 140 and the remainder of thesheath 102 removed. The first and second image sensors 106 and 108 aresubstantially identical and are mounted on a common sensor circuitsubstrate 200. The sensor circuit substrate 200 is accommodated within abore of the sensor housing portion 140 at the end of the sheath 102.Each of the image sensors 106 and 108 include a plurality of lightsensitive elements or pixels. In one embodiment, the image sensors 106and 108 may be implemented using a CMOS image sensor such as the OH02A10available from OmniVision of Santa Clara, USA. The OH02A10 image sensorhas 1.4 μm square pixels in a 1920×1080 array and the sensor has a ⅙inch (i.e. 4.23 millimeters across its diagonal). Each image sensor 106and 108 has associated imaging optics 202 and 204 disposed in front ofthe respective image sensors and configured to capture light from theobject field 110 to form images on the respective image sensors. TheOH02A10 image sensor is capable of a frame rate of 60 frames per second(fps) at full 1080p scan resolution thus providing high resolution videoimages.

In this embodiment, the sensor circuit substrate 200 on which the firstand second image sensors 106 and 108 are mounted is connected to aconnector circuit substrate 206 via a multiple pin connector 210.Referring to FIG. 2B, the sensor circuit substrate 200 is shownseparated from the connector circuit substrate 206. The connector 210includes a connector portion 210′ mounted on the connector circuitsubstrate and a corresponding connector portion 210″ mounted on a rearsurface of the sensor circuit substrate 200.

Referring to FIG. 2C, the wired signal line 118 is connected to a rearsurface 212 of the connector circuit substrate 206. In this embodiment,the connection is formed by directly soldering individual conductors 214in the wired signal line 118 at solder pads 216 on the rear surface 212of the connector circuit substrate. Signals are routed to and fromunprocessed digital data outputs of the respective first and secondimage sensors 106 and 108 via the sensor circuit substrate 200, theconnector 210, and the connector circuit substrate 206, for transmissionover the wired signal line 118 to the processing circuitry 122. Thewired signal line 118 generally includes a plurality of conductors 214,including conductors for supplying power to the image sensors 106 and108, conductors for transmitting image sensor control signals and aclock synchronization signal to the image sensors, and conductors thatact as signal transmission lines for transmitting image signals to theprocessing circuitry 122 via the wired signal line. In the embodimentshown, the sensor circuit substrate 200 includes only passive electroniccomponents 210 such as decoupling capacitors. In the embodiment shown,the only active components on the sensor circuit substrate 200 are theimage sensors 106 and 108. In other embodiments, the sensor circuitsubstrate 200 or connector circuit substrate 206 may include additionalsignal conditioning circuitry for conditioning the signals to betransmitted to the processing circuitry 122 via the wired signal line118.

Referring to FIG. 2D, in this embodiment the sensor circuit substrate200, connector circuit substrate 206, and wired signal line 118 areenclosed by a graphene sheet, a portion of which is shown at 218. Thegraphene sheet 218 extends into the bore 116 of the tube 134 and iswrapped about the sensor circuit substrate 200. The graphene sheet 218channels heat away from the image sensors 106 and 108 along a length ofthe wired signal line 118. Graphene, having a high level of thermalconductivity, is effective at channeling the heat from the image sensors106 and 108 along the sheath 102 away from a portion of the apparatus100 that will be inserted into the patient's body cavity. Operation ofimage sensors and/or processing circuitry can cause a temperatureincrease, which if occurring at the distal end of the sheath 102 mayaffect sensitive tissues at the site of surgical operations.

In some embodiments the removal of heat by the graphene sheet 218 mayreduce the temperature within the housing portion 140 of the sheath 102to a point where condensation may form on the imaging optics 202 and 204associated with the first and second image sensors 106 and 108. The bodycavity temperature of the patient will typically be somewhere in theregion of 37° C. and it would be desirable that the sensor housingportion 140 remain above this temperature to prevent condensationforming. Referring to FIG. 2D, in the embodiment shown, a heatingelement (or heater) 220 is provided at the distal end 104 of the sheath102. The heating element 220 may be wrapped around the sensor circuitsubstrate 200 under the graphene sheet 218. Suitable heating elementsincluding resistive traces formed on a flexible kapton sheet areavailable from Heatact Super Conductive Heat-Tech Co. Ltd of Taiwan orEmbro GmbH, of Germany. The heating element 220 includes a pair ofcontact pads 222 that may be connected to receive a heating current viaa pair of conductors that form part of the line 118. In one embodiment,a temperature at the first and second image sensors 106 and 108 may bemonitored and the heating current supplied to the heating element 220may be increased to heat the distal end 104 apparatus 100 when a risk ofcondensation is detected.

A schematic diagram of the processing circuitry 122, connector circuitsubstrate 206, and sensor circuit substrate 200 is shown in FIG. 3. Thefirst and second image sensors 106 and 108 are packaged for solderconnection to the sensor circuit substrate 200 via a number ofconnection points that provide power and control signals to the sensorand read out image data. In the case of the Omnivision OH02A10 sensor,the package is a chip on polymer (CIP) package having 32 connections.The sensor circuit substrate 200 provides solder pads for soldering thesensors to the substrate such that the sensors can be mounted inalignment with each other and at a fixed lateral offset or stereoscopicseparation distance. The sensor circuit substrate 200 further includesinternal traces and via connections that route the necessary connectionson the sensor to the connector portion 210″. The connector circuitsubstrate 206 similarly includes traces and vias that route theconnections from the connector portion 210′, through the substrate, andout to the solder pads 216 on the rear surface 212 of the connectorcircuit substrate (shown in FIG. 2C).

As disclosed above, in this embodiment the sensor circuit substrate 200and connector circuit substrate 206 only route connections between theimage sensors 106, 108 and the wired signal line 118 and there is noactive circuitry other than the image sensors mounted on these circuitsubstrates. In the embodiment shown, the sensor circuit substrate 200and connector circuit substrate 206 are separate substrates, whichfacilitates separate fabrication and handling of the sensor circuitsubstrate 200 for protection of the sensitive CMOS image sensors 106,108. In other embodiments, the sensor circuit substrate 200 andconnector circuit substrate 206 may be fabricated as a single circuitsubstrate or the image sensors 106, 108 may be otherwise mounted andconnected.

The wired signal line 118 includes the plurality of individualconductors 214 that extend between the solder pads 216 and the strip ofcircuit substrate 124, which connects to the multiple pin connector 126on the processing circuitry 122. As disclosed above, image data fromeach of the first and second image sensors 106 and 108 are transmittedas unprocessed digital data signals to the processing circuitry 122 viathe wired signal line 118. In the example of the Omnivision OH02A10sensor, the unprocessed digital data signals comply with the MIPI CSI-2transmission protocol, which is a camera serial interface protocoladministered by the Mobile Industry Processor Interface (MIPI) Alliance.Other unprocessed data signal or raw image data protocols may beimplemented depending on the selection of image sensors. Video signalsmay be transmitted using a variety of signal protocols such as Serialdigital interface (SDI), Serial Peripheral Interface (SPI), I²C(Inter-Integrated Circuit), RGB video, or Low-voltage differentialsignaling (LVDS), some of which may be employed to transmit the videosignals along the wired signal line 118.

In the embodiment shown in FIG. 3, the wired signal line 118 includestwo twisted pairs of conductors for implementing two MIPI data lanes foreach image sensor 106, 108. For the first (right hand side) image sensor106, MIPI data lane 0 and MIPI data lane 1 are implemented, while forthe second (left hand side) image sensor 108, MIPI data lane 0 and MIPIdata lane 1 are implemented. The wired signal line 118 also includes atwisted pair of conductors for transmitting a clock associated with theMIPI signals for each of the sensors 106 and 108. In other embodiments,a single MIPI data lane or more than two MIPI data lanes may beimplemented for each image sensor 106, 108. The MIPI CSI-2 protocol isgenerally used for short distance transmission between an imaging sensorand processing circuitry within in an imaging device, but in theapparatus 100 the distance over which transmission occurs issignificantly increased. The wired signal line 118 also includes severalpower and ground conductors for delivering operating power to each ofthe sensors 106 and 108. The wired signal line 118 further includes acoaxial conductor for transmitting a synchronization clock signalbetween the processing circuitry 122 and each of the first and secondimage sensors 106 and 108. The wired signal line 118 also includes atwo-wire inter-integrated circuit (I²C) control line for carrying imagesensor control signals between the processing circuitry 122 and each ofthe image sensors 106 and 108. The I²C control may be implemented usinga serial data line (SDA) and a serial clock line (SCL) to control eachof the image sensors 106 and 108. The image sensor control signals areused to set up and control parameters for image capture at the chiplevel.

An imaging apparatus used to generate views inside a patient's bodycavity will generally have a length of at least 300 millimeters orgreater between the distal end and the proximal end. In the example ofthe apparatus 100 shown in FIG. 1, the overall length is about 900 mm.The Omnivision OH02A10 sensor has 2,073,60 pixels (1920×1080) and anoutput format of 10 bit raw RGB data representing each pixel intensity.A single frame thus amounts to 20,736,000 bits of data, which at a framerate of 60 frames per second represents a data rate of 1.24 gigabits persecond for each image sensor. In the apparatus 100, there is arequirement to continuously transmit well in excess of 2 gigabits persecond. This high data rate must be carried along the wired signal line118 per sensor without significant signal degradation since imagedropout in a medical device such as a surgical system is generallyconsidered to be unacceptable. Data rates of well over 1 gigabit persecond are not generally regarded to be successfully transmissible overMIPI CSI-2 data lanes when the length of transmission exceeds about 30millimeters and thus the 900 mm transmission would be generally avoided.

The processing circuitry 122 is connected to the connector circuitsubstrate 206 via the wired signal line 118 at the multiple pinconnector 126. In this embodiment, the processing circuitry 122 includessignal conditioning blocks 300 and 302 that receive and condition theunprocessed data signals (i.e. the MIPI data 0 and data 1 lane signalsfrom the respective image sensors 106 and 108). In one embodiment, theMIPI data lane signals may be passed through a circuit that boosts thepotentially weakened signals and to compensate for any degradation ofthe bits in each data stream. In this embodiment, the signalconditioning blocks 300 and 302 are implemented in the processingcircuitry 122, which is disposed after the unprocessed data signals havebeen transmitted along the length of the wired signal line 118. In otherembodiments, signal conditioning may additionally or alternatively beperformed partway along the sheath 102 of the apparatus 100 in-line withthe wired signal line 118. Alternatively, in other embodiments, signalconditioning may be performed at the sensor circuit substrate 200 orconnector circuit substrate 206 should it be necessary to further extendtransmission distance for the image signals. In other embodiments,signal conditioning functions may be omitted where the signaldegradation during transmission over the wired signal line 118 is not ofissue.

The conditioned signals are then passed by the signal conditioningblocks 300, 302 to respective image signal interface blocks 304 and 306for conversion into video signals suitable for transmission over alonger distance to a host system. The image signal interface blocks 304and 306 may each be implemented using a field-programmable gate array(FPGA) that performs a signal format conversion between the unprocessedsignal format and a format suitable for transmission to a host system.The FPGA generally combines the MIPI data lane 0 and 1 streams andformats the signal into a video signal that can be transmitted over agreater distance to a host system and/or displayed on a 3D capabledisplay. The processing circuitry 122 further includes ports 308 and 310for connecting the apparatus 100 to enable a host system to receive theprocessed and formatted video image signals for display and/or furtherprocessing. The image signal interface blocks 304 and 306 may optionallyperform other image processing functions on the signals such asfiltering, white balance, color control and correction, etc.

In one embodiment, the image signal interface blocks 304 and 306 may beconfigured to produce serialized signals that comply with a flat paneldisplay (FPD)-Link signal transmission protocol, which can betransmitted to a host system over a coaxial signal line via the ports308 and 310. For example, an interface implementing the FPD-Link IIIupdate to the protocol is able to transmit the video data signals andalso embed clock signals and a bidirectional communication channel onthe same signal line.

In another embodiment, the image signal interface blocks 304 and 306process and format the image data from each image sensor 106 and 108into a 3G serial digital interface (SDI) serial data stream and theports 308 and 310 are implemented as BNC connectors that connect tocoaxial cables for carrying the first (right) and second (left) imagesensor SDI serial data streams to the host system or display. SDI is afamily of digital video interfaces commonly used for transmittingbroadcast-grade video.

The processing circuitry 122 also receives a power feed at a connector312 and further includes a power and control block 314, which isconfigured to receive the power feed and to supply power to the imagesensors 106 and 108. The power and control block 314 also provides acontrol interface for sending imaging commands between a host system andthe image sensors 106 and 108. In the embodiment shown, where the imagesignal interface blocks 304 and 306 implement the FPD-link III protocol,the bidirectional communication channel may be exploited to transmitimage sensor control commands to the first and second image sensors 106and 108 via the ports 308 and 310. In this case, the image signalinterface blocks 304 and 306 are further configured to detect imagesensor command signals on the bi-directional communication channel andto forward these command signals to the power and control block 314. Thepower and control block 314 acts as an interface for transmission of thecommands to the respective image sensors 106 and 108 via the PC controlsignal conductors within the wired signal line 118.

Referring to FIG. 4, an insertion device for use with a robotic surgeryapparatus (not shown) is shown at 400. The insertion device 400 includesan insertion section 402, which in turn includes first and second camerachannels 404 and 406 for receiving a camera, such as the imagingapparatus 100. The insertion section 402 also includes one or moreinstrument channels (in this case instrument channels 408 and 410) thatextend along at least a portion of the insertion section 402. Theinsertion section 402 is inserted into a body cavity of a patient andinstruments (manual laparoscopic or robotic) are inserted through theinstrument channels 408 and 410 to perform surgical operations withinthe body cavity. The insertion device 400 also includes a housing 414attached to the insertion section 402. The housing 414 includes apassage 416 configured to permit at least a portion of the sheath 102 ofthe apparatus 100 to pass through the housing into the first camerachannel 404 and to exit the first camera channel. The imaging apparatus100 acts as a primary camera for producing 3D stereoscopic images of thebody cavity. The second camera channel 406 receives a secondary camera412, which provides 2D image data of a surgical site within the bodycavity to facilitate insertion into the surgical site of surgicalinstruments through the instrument channels 408 and 410. The secondarycamera 412 generates images of the body cavity of the patient prior tothe apparatus 100 being inserted to provide the surgeon with a view ofthe surgical site for insertion of the instruments and the primarycamera apparatus 100. The insertion device 400 is described in moredetail in commonly owned patent application entitled “SYSTEMS, METHODS,AND APPARATUSES FOR CAPTURING IMAGES DURING A MEDICAL PROCEDURE”, filedon Jun. 21, 2019 as U.S. patent application Ser. No. 16/449,095 and incommonly owned U.S. Pat. No. 10,398,287 entitled “CAMERA POSITIONINGSYSTEM, METHOD, AND APPARATUS FOR CAPTURING IMAGES DURING A MEDICALPROCEDURE”, both of which are incorporated herein by reference in theirentirety.

An embodiment of the secondary camera 412 is shown in FIG. 5. Referringto FIG. 5, in the embodiment shown, the secondary camera 412 includes animage sensor 500 oriented to capture high definition images of an objectfield 502. The image sensor 500 is mounted at a distal end 504 of anelongate strip of circuit substrate 506. The circuit substrate 506includes a plurality of conductors 508 extending along the circuitsubstrate to a proximal end 510. The secondary camera 412 includes aprocessing circuit substrate 512, shown in greater detail in an insertin FIG. 5. The processing circuit substrate 512 includes a multiple pinconnector 514, which is configured to receive and connect to the circuitsubstrate 506. The circuit substrate 506 includes a plurality ofconductors 508 on the elongate circuit substrate 506 that form aconnector at the proximal end of the strip. In some embodiments, twoimage sensors may be mounted at the distal end 504 to capturestereoscopic image data.

The distal end 504 of the secondary camera 412 is shown in greaterdetail in an insert in FIG. 5. In the embodiment shown, the image sensor500 is connected to a sensor circuit substrate 516, which in turn isconnected to the elongate circuit substrate 506. In one embodiment theconnection between the sensor circuit substrate 516 and the elongatecircuit substrate 506 may be a directly soldered connection. The imagesensor 500 may produce an unprocessed digital data signal representingimages captured from the object field 502 as described above inconnection with the apparatus 100. The image sensor 500 is connected viathe sensor circuit substrate 516 to the conductors 508 on the circuitsubstrate 506. The elongate strip of circuit substrate 506 is sized forinsertion through a narrow conduit such as the second camera channel 406of the insertion device 400 shown in FIG. 4. In some cases, the width(or diameter) of the conduit can be about 4 millimeters. In someembodiments, the secondary camera 412 may be removably insertablethrough the secondary camera channel 406. As an example, this wouldallow replacement of the secondary camera with another camera capable ofcapturing non-visible light (such as infrared light or the like).Alternatively the secondary camera may be replaced with a surgicalinstrument or other accessory.

The processing circuit substrate 512 includes processing circuitry 518configured to receive and process the unprocessed digital data signalfrom the image sensor 500 to produce a video signal suitable fortransmission to a host system for driving a display. In one embodiment,the elongate strip of circuit substrate 506 has a length of about 20centimeters and a width of about 4 millimeters. In other embodiments,the circuit substrate 506 may have a length of greater than 20centimeters and may be wider or narrower than 4 millimeters. Theelongate circuit substrate 506 and image sensor 500 may be fabricated asa module that facilitates insertion of the proximal end 510 through thesecond camera channel 406 in the insertion section 402 of the insertiondevice 400. Once inserted, the camera module may be connected to themultiple pin connectors 514 at the proximal end.

The above embodiments of both the 3D stereoscopic primary imagingapparatus 100 and the secondary camera 412 provide for separationbetween the image sensors and the processing circuitry whilefacilitating relatively convenient handling during manufacture andsubsequent use. The transmission of unprocessed raw image data from thesensor chips to the processing circuitry over a longer than conventionaldistance separates heat sources that would otherwise be in closeproximity to the portions of the imaging apparatus 100 and secondarycamera 412 that are inserted into the body cavity of the patient.

While specific embodiments have been described and illustrated, suchembodiments should be considered illustrative only and not as limitingthe disclosed embodiments as construed in accordance with theaccompanying claims.

What is claimed is:
 1. A stereoscopic imaging apparatus for use in arobotic surgery system, the apparatus comprising: an elongate sheathwith a bore extending therethrough, the sheath terminating in a distalend sized for insertion into a body cavity of a patient; first andsecond image sensors adjacently mounted at the distal end of the sheathand oriented to capture high definition images of an object field fromdifferent perspective viewpoints for generating three-dimensional imageinformation, each of the first and second image sensors configured toproduce an unprocessed digital data signal representing the capturedimages; a wired signal line configured to transmit the unprocesseddigital data signals from each of the first and second image sensorsalong the sheath to a proximal end thereof; and processing circuitrydisposed at the proximal end of the sheath and connected to the wiredsignal line to receive the unprocessed digital data signals from each ofthe first and second image sensors, the processing circuitry configuredto perform processing operations on each of the unprocessed digital datasignals to produce respective video signals for transmission to a hostsystem and driving a display configured to display three-dimensionalinformation.
 2. The apparatus of claim 1 wherein a bit rate of each ofthe unprocessed digital data signals has is higher than about 1 gigabitper second.
 3. The apparatus of claim 1 wherein each of the first andsecond image sensors include at least about 2,000,000 pixels.
 4. Theapparatus of claim 3 wherein at least unprocessed digital data signalcomprises 10 bit pixel intensity values read out from the pixels of therespective first and second image sensors.
 5. The apparatus of claim 1wherein at least one unprocessed digital data signal comprises a signalin accordance with a Mobile Industry Processor Interface (MIPI) CameraSerial Interface protocol and wherein the length of the sheath isgreater than 30 millimeters.
 6. The apparatus of claim 5 the length ofthe sheath is at least about 800 millimeters.
 7. The apparatus of claim5 wherein the wired signal line comprises a plurality of individualconductors including: conductors for implementing at least one MobileIndustry Processor Interface (MIPI) data lane for each image sensor;conductors for transmitting a synchronization clock signal between theprocessing circuitry and the first and second image sensors; and atleast two conductors for carrying image sensor control signals.
 8. Theapparatus of claim 1 wherein the first and second image sensors aremounted on a sensor circuit substrate disposed within the bore of thesheath and wherein the wired signal line comprises a plurality ofindividual conductors connected via the sensor circuit substrate to theunprocessed digital data outputs of the respective first and secondimage sensors.
 9. The apparatus of claim 8 wherein the plurality ofindividual conductors of the wired signal line are connected at theproximal end to a strip of the sensor circuit substrate sized to passthrough the bore of the sheath, the strip of the sensor circuitsubstrate including a multiple pin connector for connecting to acorresponding multiple pin connector on a circuit substrate associatedwith the processing circuitry.
 10. The apparatus of claim 8 furthercomprising a graphene sheet within the bore of the sheath, the graphenesheet being in thermal communication with the sensor circuit substrateand wrapped around at least a portion of a length of the wired signalline to channel heat away from the distal end of the sheath.
 11. Theapparatus of claim 10 further comprising a heater disposed at the distalend of the sheath and configured to selectively heat the distal end ofthe sheath to maintain the distal end of the sheath at a temperaturethat prevents formation of condensation.
 12. The apparatus of claim 1further comprising signal conditioning circuitry configured to conditionthe unprocessed digital data signals for transmission, the signalconditioning circuitry comprising at least one of: conditioningcircuitry positioned at the distal end of the sheath between each of thefirst and second images sensors and the wired signal line; conditioningcircuitry located partway along the sheath in-line with the wired signalline; or conditioning circuitry configured to re-condition the receivedunprocessed digital data signals prior to performing processingoperations on the signals.
 13. The apparatus of claim 1 wherein theprocessing circuitry is configured to convert each of the unprocesseddigital data signals into a serial digital interface (SDI) video signalfor transmission to the host system.
 14. The apparatus of claim 1wherein the processing circuitry is configured to convert each of theunprocessed digital data signals into a flat patent display (FPD) linkvideo signal for transmission to the host system.
 15. The apparatus ofclaim 1 wherein the sheath comprises one of a rigid sheath or a flexiblesheath.
 16. The apparatus of claim 1 wherein the sheath comprises aflexible articulating portion which, when actuated by the host system,facilitates movement of the distal end of the sheath within the bodycavity of a patient to orient the image sensors for image capture. 17.The apparatus of claim 1 further comprising a plurality of opticalfibers extending through the sheath and terminating at the distal end,the plurality of optical fibers configured to channel light from adistally located light source for illuminating the object field.
 18. Theapparatus of claim 17 wherein the first and second image sensors aremounted on a sensor circuit substrate sized to occupy a central portionof the bore of the sheath and wherein the plurality of optical fibersterminate at one or more regions between the sensor substrate and thesheath at the distal end of the sheath.
 19. The apparatus of claim 1wherein the sheath has a generally circular cross section.
 20. Theapparatus of claim 19 wherein an outside diameter of the sheath is lessthan about 10 millimeters.
 21. The apparatus of claim 1 wherein each ofthe image sensors include imaging optics disposed in front of therespective faces of each of the image sensors and configured to capturelight from the object field to form an image on the respective imagesensor.
 22. An imaging apparatus comprising: an image sensor oriented tocapture high definition images of an object field and configured toproduce an unprocessed digital data signal representing the capturedimages; and an elongate strip of circuit substrate sized for insertionthrough a conduit, the image sensor being mounted at a distal end of thecircuit substrate and connected to a plurality of conductors extendingalong the elongate circuit substrate to a proximal end thereof, theproximal end including a multiple pin connector for connecting to acorresponding multiple pin connector on a processing circuit substrate,the processing circuit substrate including processing circuitryconfigured to receive and process the unprocessed digital data signalfrom the image sensor to produce a video signal for transmission to ahost system and driving a display.
 23. The apparatus of claim 22 whereina length of the elongate strip of circuit substrate is at least about 20centimeters and a width of the elongate strip of circuit substrate isless than about 4 millimeters.
 24. An insertion device for a roboticsurgery apparatus, the insertion device comprising: an insertion sectioncomprising first and second camera channels and at least one instrumentchannel extending along at least a portion of the insertion section, thefirst camera channel configured to facilitate insertion and removal fromthe insertion section of the sheath and the first and second imagesensors of claim 1 for use as a primary camera, the second camerachannel configured to enclose the image sensor and elongate strip ofcircuit substrate of claim 18 for use as a secondary camera, and the atleast one instrument channel configured to permit insertion and removalof at least one surgical instrument from the insertion section; and ahousing attached to the insertion section, the housing comprising apassage configured to permit at least a portion of the primary camera topass through the housing into the first camera channel and exit thefirst camera channel, the housing configured to be removably attached tothe robotic surgery apparatus; wherein the secondary camera isconfigured to provide image data of a surgical site to facilitateinsertion into the surgical site of at least one of the at least onesurgical instrument or the primary camera.